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(57) Abstract 

An interactive interface facilitates the setting of preferences and other programmable parameters of an appliance. The interface is 
hosted by a server on a global computer network. The appliance owner initiates a connection to the server and is presented with a graphical 
user interface for setting the preferences and features of the appliance. Once the desired settings have been made, they are downloaded to 
the appliance either directly from the server or the appliance owner's computer or indirectly using a portable transfer device. 
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METHOD AND APPARATUS FOR SETTING PROGRAMMABLE 
FEATURES OF AN APPLIANCE 

BACKGROUND OF THE INVENTION 

1. FIELD OF THE INVENTION 

This invention relates generally to the field of setting programmable features of 
an appliance. More particularly, the invention provides a method and apparatus for 
conveniently setting various programmable features of an appliance using a graphical 
user interface accessed with a computer via a global computer network. 

2. PRIOR ART 

The advent of microprocessors and other miniaturized electronics has facilitated 
the implementation of increasingly complex functions in home and office appliances. 
Typically, a relatively complex operator interface is required in order to invoke the 
various functions that are available. For example, home electronic devices such as 
televisions, VCRs, stereo receivers and the like are typically provided with sophisticated 
remote control devices. Such remote control devices have a large number of individual 
buttons that are used to directly control features of an appliance and/or that are used to 
navigate through on-screen menus. Because of the sophistication and complexity of the 
controls, owner's manuals for appliances are becoming increasingly voluminous and 
difficult to comprehend. 

Due to the growing complexity of modern appliances, many of the available 
features are never utilized by consumers, even as competition in the marketplace drives 
the proliferation of such features. A number of solutions have been proposed for 
making appliances easier to control and generally more "user friendly". For example, 
U.S. Patent No. 5,553,123 issued to Chan, et al. discloses a method for downloading 
set-up data via a telephone to an appliance controller. A user first initiates a telephone 
call to a remote site having a computer. The user communicates certain background 
information to the remote site, and set-up data is then downloaded via the telephone 
connection. The earpiece of the telephone is held in proximity to a microphone built 
into the appliance controller in order to receive the downloaded data. Upon receipt of 
the data, the controller is configured to operate the appliance. 
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U S Patent No. 5,600,71 1 issued to Yuen discloses an apparatus and methods 
for providing initializing settings to an applianee. When a user wishes to mtt.al.ze the 
settings of an applianee, the user initiate, a telephone connection with a remote sue. 
The r mote site then dow„,oads a sequence of commands for initializing the settmgs .» 
2 appUance over the teiephone connection. A remote control device for the appltance 
the sequence of commands and stores tttem in an interna, memory The remote 
control device is then aimed at the appliance and the user enters a command to transfer 
the stored sequence of commands to the appliance, thereby initializing the settmgs. 

U S Patent No. 5,141,756 issued to Levine discloses a method of initializing a 
programmable control device, such a, a remote controller for a video cassette recorder. 
The device is programmed by connecting it to a telephone system, dialing a remote 
initializing center preferably employing a computer, and providing the computer wtth 
information as to the environment of the control device by using touch tone keys to 
,espond to audio inquiries transmitted by the computer. The computer then transmtts 
the initializing program for loading into the memory of the control devtce. 

U s Patent No. 5,774,063 issued to Barry, et al. discloses a method and 
apparatus for remote control of elecronic devices from a computer. A transducer, such 
Tan infrared transmitter, is coupled .o acomputer and aimed a, an electron* devtce to 
be controlled. An application program nmning on the computer generates approbate 
signals for control of the electronic device. 

U S Patent No. 5,815,086 issued to Ivie, et al. discloses a method and apparatus 
for communicating commands to e.ec.rical appliances from remote locations. Vartous 
appliances within a structure, such as a house, are coupled to a signal-conducting bus, 
such as the AC power wiring bus of the structure. Appliance commands are tssued over 
the bus from a central transmitter. Appliances may bo directly coupled to the bus ot 
nray receive commands via an infrared signal from an infrared transmitting devtce 
coupled to the bus. A handheld control device may be supplied for controlling the 
various appliances, in which case, reeeivers for the handheld control device are coupled 
to the bus in various parts of the structure. 

U S Patent No. 5,819,294 issued to Chambers discloses a programmable 
universal remote controller. A programming device is coupled to a computer and 
receives signals from conventional remote controllers. The programming dev.ee 
correlates the received signals with a database of stored signals used by various 
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appliance manufacturers. The programming device then sends a complete set of 
appropriate control signals to the programmable universal controller. 

U.S. Patent No. 5,228,077 issued to Darbee discloses a universal remote 
controller that may be programmed from a remote location. The remote controller 
receives programming data via a video or telephonic data transmission system. 

U.S. Patent No. 5,488,571 issued to Jacobs, et al. discloses a system for 
transferring data from a video display monitor of a personal computer to a portable 
information device such as an appointment scheduling device. The video display is 
modulated to transmit data to an opto-electronic receiver in the portable information 
device. 

Microsoft Corporation has introduced a cordless phone having programmable 
functions controlled by a personal computer. The base station of the phone is coupled 
to the serial port of a computer and application software is installed on the computer to 
control operation of the phone. 
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SUMMARY OF THF INVENTION 

The present invention provides methods and apparatus for setting preferences 
and other parameters of an appliance. In preferred embodiments of the invention, a user 
initiates a connection to an interactive site on a global computer network. The site hosts 
a graphical user interface with which preferences and other parameters of an appliance 
may be set by the user. In some embodiments, set-up data for the appliance may be 
downloaded directly to the appliance from the user's computer or the interactive sue. In 
other embodiments, set-up data for the appliance is downloaded from the user's 
computer or the interactive site to a transfer device where it is temporarily stored. The 
transfer device, which is easily portable, is taken by the user to the appliance, and the 
set-up data is downloaded from the transfer device to the appliance. Since the appliance 
itself does not require a user interface for set-up procedures and programming, the 
appliance can be smaller, cheaper and lighter without sacrificing any functionality. In 
addition, the need for a printed user's manual is largely obviated since all of the 
information normally contained in such a manual can be obtained from the interacUve 



site. 



WO 00/43870 



5 



PCT/US00/01541 



BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a functional block diagram of a first embodiment of the invention 
wherein an appliance receives data directly from a local computer. 

Figure 2 is a functional block diagram of a second embodiment of the invention 
wherein an appliance receives data from a local computer via a transfer device. 

Figure 3 is a functional block diagram of a transfer device as shown in Figure 

2. 

Figure 4 illustrates a graphical user interface suitable for setting programmable 
features of a thermostat. 

Figure 5 is a functional block diagram of a third embodiment of the invention 
wherein an appliance receives data directly from an interactive site server. 

Figure 6 is a functional block diagram of a fourth embodiment of the invention 
wherein an appliance receives data from an interactive site server via a transfer device. 

Figure 7 illustrates luminance modulation for transferring decimal data digits. 

Figure 8 illustrates luminance modulation with dithering encoding. 

Figure 9 illustrates luminance modulation with irregular graphic patterns. 

Figure 10 illustrates bi-color phase modulation. 

Figure 11 illustrates video bar code modulation. 

Figure 12 illustrates a self-clocking data encoding scheme for use with the 
present invention. 

Figure 13 is a functional block diagram of a receiving device suitable for use 
with the present invention. 

Figure 14 illustrates a display screen having a portion thereof for data transfer. 

Figure 15 is a plot of CRT photoresponse of a prototype system constructed in 
accordance with the present invention. 
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Figure 16 is a plot of a calibration sequence used in the prototype system. 

Figure 17 illustrates an embodiment of the invention employed to provide a 
visual index to the contents of a storage medium. 

Figures 18-20 illustrate the transfer of a virtual companion from a personal 
computer to a handheld device in accordance with an embodiment of the present 
invention. 
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DETAILED DESCRIPTION OF THE INVENTION 

In the following description, for purposes of explanation and not limitation, 
specific details are set forth in order to provide a thorough understanding of the present 
invention. However, it will be apparent to one skilled in the art that the present 
invention may be practiced in other embodiments that depart from these specific details. 
In other instances, detailed descriptions of well-known methods and devices are omitted 
so as to not obscure the description of the present invention with unnecessary detail. 

The present invention finds application with a wide variety of home and office 
appliances. Some categories of appliances in which the invention may be utilized 
include clocks, telephones, televisions, television set-top decoders, video recorders, 
audio and video entertainment system components, refrigerators, conventional ovens, 
microwave ovens, dishwashers, irrigation systems, global positioning satellite (GPS) 
receivers, automobile heating, ventilating and air conditioning (HVAC) systems, 
automobile sound systems, home security systems, home HVAC systems, home master 
control systems, facsimile machines, copiers, cameras, postage meters, etc. 
"Programmable features" refer to any appliance features that may be altered. These 
include, for example, initialization or set-up parameters, stored data (e.g., telephone 
speed dial numbers or GPS receiver database) and internal software. Specific examples 
are given below to illustrate operation of the invention. However, it will be understood 
that the invention has general applicability to appliances of all types and to all types of 
programmable features within such appliances. 

The invention also has applicability apart from setting programmable features of 
appliances. For example, the invention may be employed to purchase pay-per-view 
programming at an interactive web site. An authorization code may then be 
downloaded into the transfer device of the invention and transferred to a TV set-top box 
so that the purchased program will be "descrambled". This approach eliminates the 
telephone connection required for most TV set-top boxes with pay-per-view capability. 

Another potential application for the invention is as a programmable "token" 
which may be used to facilitate a wide variety of transactions. For example, a consumer 
product manufacturer may offer discounts on certain of its products at its web site. 
Authorization to receive the discount may be downloaded into the transfer device of the 
invention and the transfer device may then be taken to a retailer. The transfer device is 
then used to transmit the discount authorization to a receiving terminal at the retailer. 
Ideally, the terminal would also have the ability to modify the stored contents of the 
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transfer device so that the discount authorization could be cancelled once the discount is 
given This same "token" approach can also be applied to pre-paid purchase 
transactions; reservations at restaurants, hotels, parks, etc.; entry authorization to 
entertainment venues or other secured areas and similar situations in which a 
conveniently transported authorizing token serves as an extended communication link 
from a computer system. 



1. 



SPECIFIC EMBODIMENTS OF THE INVENTION 



A first embodiment of the invention is illustrated in Figure 1. An appliance 10 
receives set-up data from a local computer 12. In a typical application, local computer 
12 is a general purpose personal computer of the type now widely found in homes and 
offices. Details of computer 12 are not particularly relevant to the invention and are not 
shown Typically, computer 12 will comprise, at a minimum, a processing unit, a 
keyboard and a display. Additional input devices, such as a mouse or other pointing 
device and output devices, such as a printer, may also be included as part of computer 



12. 



Local computer 12 is coupled to a remote interactive site server 14 by a 
telecommunications link. In a typical embodiment of the invention, interactive site 
server 14 would be accessible via the World Wide Web. Other appropriate means for 
connecting computer 12 to server 14 could also be employed. Server 14 contains 
programming for interactively setting the programmable features of appliance 10. 
Preferably server 14 presents to the owner of appliance 10, via computer 12, a 
graphical user interface that is tailored to appliance 10 and the programmable features 
thereof. Such interface can be thought of as a "virtual appliance". This can be better 
understood from the discussion of Figure 4, below. 

In the embodiment illustrated in Figure 1, appliance 10 is coupled directly to 
local computer 12. This embodiment is best suited for portable appliances that may be 
conveniently carried to the computer for set-up. The coupling between appliance 10 
and computer 12 may be one-way from the computer or two-way. One-way 
communication may be accomplished optically by providing appliance 10 with an 
optical sensor and modulating the display of computer 12 utilizing one or more of the 
techniques described below. Other communication techniques can be employed using 
audio magnetic, inductive, infrared, or radio frequency coupling. Two-way 
communications are most conveniently established by connection to a serial port of 
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computer 12. Naturally, this type of connection is not convenient for some appliances, 
but is particularly well-suited to portable appliances that may require a large amount of 
data. For example, loading data into a pocket organizer or similar type of personal 
digital assistant can be most conveniently accomplished with a serial port connection in 
the configuration illustrated in Figure 1. 

Some types of appliances can be readily adapted to utilize existing components 
for establishing communications with computer 12. For example, electronic cameras 
inherently possess optical sensors that can be used to sense modulation of a computer 
display screen or other light source. The primary imaging path of the camera may be 
used in the case of video cameras and digital still cameras. This simply requires the 
addition of circuitry and/or software to decode the modulation and store the appropriate 
set-up parameters. Alternatively, the receiver of the camera's focusing range finder 
may be used as the optical sensor. 

In order to provide the appropriate interface for programming the features of 
appliance 10, server 14 preferably receives data from the appliance manufacturer. Such 
data may be received periodically as new model appliances are released by the 
manufacturer or may be obtained by server 14 in real time with a dial-up connection to 
the manufacturer. The latter approach offers the advantage of insuring that the most 
recent product information is available to server 14. One method of insuring that the 
appropriate information for appliance 10 is obtained by server 14 is to prompt the 
appliance owner to input the serial number of the appliance at computer 12. This need 
only be performed once, since the serial number can thereafter be stored in computer 12 
and/or server 14 for use in subsequent programming of the same appliance. Warranty 
registration for the appliance may be conveniently performed during this same 
procedure. 

An optional aspect of the invention is the ability of server 14 to provide valuable 
feedback to the appliance manufacturer. During appliance set-up operations, server 14 
collects information concerning consumer's use of product features that can be useful in 
product marketing and new product design. The link between server 14 and the 
appliance manufacturer also facilitates new marketing opportunities. The manufacturer 
can readily target advertising to identified purchasers of its products. Also, the 
manufacturer can offer accessories and related products for appliance 10. Such offers 
may be integrated with the set-up interface or may be directed to the appliance owner 
separately by email or conventional mail. It should be appreciated that the invention 
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can facilitate waminty registration. Since the appliance owner is already 
communicating with server ,4 to se, provable features of the ^ 
simrte matter to collect the additional information necessary to complete warranty 
:JaL and, if desired, to provide additional demographic data to the manufacture, 

Freure 2 illustrates an alternative embodiment of the invention. This 
embodiml is similar to that of Figure 1, except that programming data is prov.ded to 
aoZTe 10 by a transfer device 16. This transfer device receives the programing 
£Z Z compute r 12 by a wired connection to computer 12 or, preferably by an 
canonic or oL wireless data >mk such as will be described nrore fu..y be.ow. 

Figure 3 is a functional block diagram of a suitable transfer device 16. At the 
hear, of dev.ce ,6 is a control electronics module 102. Data modulated on the d.splay 
screen of computer 12 is sensed by optica, detector 104 upon acttvatton of recewe 
ZZ 106 The data is demodulated by electromcs 102 and is stored m memory 108. 
oferror-fteeuansferands^geofOteda^ asui 

provided ,„ the user by means of indicator 110, which may be, for example, a light 
Z ing diode (LED). With me data loaded in memory 108, transfer dev.ce 16 may be 
o appliance 10, which preferably includes a "docking" port or transfer dev.ce 
7^ device 16 may be an integral component of appliance 10, wmc h . 
pLded to the consumer by .he appliance manufaemre, Alternately applmnce 10 
may be provided with an infrared receiver coupled * its internal control electromcs. 1 
rlofanmlmk,^^^^^ 

appUance 10. Upon actuation of transmit switch 112, me data stored m memory 108 is 
ipdarely modulated by eleconics 102 and applied to infrared — er U . 
Ind icl 110 may confirm to me user mat the data has been transmmed. Alternately, 
"on, an indicator may be provided on appliance 10 to signal rece,p, o me 
TaL Power source 116, prefembly in the form of common alkahne battery cells, 
provides electrical power to the components of device 16. 

Transfer device 16 may be configured to transfer data from the appliance back to 
te computer as well. Data from the appliance may be .oaded into the transfer device 
« of an opto-decttonic link in the same manner by which data is loaded from 
TZ^. Pmferably, however, me transfer device wil, have a direct e,ec,r,cal 
SL, « the appUance for applications invoMng 2-way commumcattous. Tra fer 
7L nto ,he computer may he accomplished in a number of ways. For example, 
lander device 16 may couple directly to a serial or paraUel input port of me computer. 
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Alternatively, transfer device 16 may include a sound transducer by which data may be 
transferred through a microphone coupled to the computer. 

Transfer device 16 may be configured in various forms. Preferably, device 16 is 
easily portable. Device 16 may be in the form of a pen or wand with optical detector 
104 and infrared transmitter 114 at one end. Transfer device 16 may also be integrated 
with a conventional remote controller for those types of appliances that are commonly 
controlled remotely. In another variation, transfer device 16 may be a removable 
module that is docked into appliance 10 as described above. In such case, 
communication between the transfer device and the appliance may be accomplished 
with a direct electrical connection through a suitable arrangement of electrical contacts. 
Transfer device 16 may, in fact, comprise the "brains" of appliance 10 in the form of a 
microprocessor or equivalent device. Aside from the ease of programming features and 
functions of the appliance, such an arrangement offers the added benefit of facilitating 
service or replacement of the appliance's electronic components in the event of 
malfunction. 

The embodiment shown in Figures 2 and 3 is particularly well-suited to 
appliances that are relatively fixed in position and that require only limited amounts of 
data. Examples of such appliances abound in the home and office. One such example 
is a thermostat for a home HVAC system. Figure 4 illustrates a graphical user interface 
for a thermostat as presented on a display of computer 12. Such interface is shown 
merely for purposes of illustration, it being understood that the particular features of the 
interface are largely a matter of design choice. The graphical user interface is 
sometimes referred to as a "virtual appliance" (a virtual thermostat in this specific 
example). This does not imply that the interface physically resembles the real 
appliance, but rather that the interface permits access, via the user's computer, to the 
programmable features of the real appliance. 

Along the top of the display shown in Figure 4 is a day strip 122. The user may 
select any one of the days with a cursor to program the thermostat settings for that day. 
Below the day strip is a temperature selector 124. Pointing at the up or down arrow 
with a mouse or other cursor positioning device, the user selects the desired 
temperature. To the right of temperature selector 124 there are a pair of time windows 
126 and 128. Using the appropriate up and down arrows, the user selects the starting 
and ending times for which the temperature selection applies. When the desired settings 
have been made, the user selects ENTER button 130 to store the selections and then 
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OT „cee<ls «o make .he next set of selections. For oonveniettce, the ending ..me las 
^may * — a„v inserted into .he sinning time window. A graph.cal 
display 132 of the seleeted temperature profile may be provided for the user s 
convenience. 

When all settings have been completed, the data is loaded into 
U which is then taken to the physical location of .he thermostat for rransfer of the data, 
stce of *e settings have been entered into computer 12, drey may be convemenuy 
.Ted^ny and/or by server ,4 for subsequent use in revising these setttngs or for 
Ing the settings in rhe event of a power failure. A printed record of dte semngs 
be made from computer 12. For some appliances, a print-out foflowmg a ~ 

1 programmed features. For example, eertain applianees may have unlabe ed 
IcUonCons for which a template may be made once selectee functions have been 
assigned to the buttons during a set-up procedure. 

I, will be appreciated that a thetmostat physically incorporating the interface 
shown in Figure 4 would be quite large and costly in comparison to conventual 
TZL This is due primarily to the relative complexity of the interface, stnce the 
"tlry to provide such flexibility of thermostat settings is actually qutte 
7 ,nd inexpensive Through use of the present invention, vrtually unltmtted 

— ■ progrLung may he accompUshed with a thermos., that is no 
"d no more costly Aran a convention* thermostat, indeed, a thermostat a S) ust 
described could easily be made the size of a postage stamp. 

Figure 5 illustrates another alternative embodiment of the invention. In this 
case data for appliance 10 is received directly from server !4 ranter man .ocal computer 
12 From the alliance owner's perspective, .he appliance programing tnterfaee ts 

rver 14 and appliance 10 may be telephonic. Appliance 10 may rncorporate a 
Zttiona. mln, in which case communications may be two-way, or may amply 
Tavl a data demodulator for one-way communication, Coupling of app.tance 10 to the 
tdecommunications network may be by a conventional RJ-1 1 
Alternatively, appliance 10 may incorporate a cordless telephone module for 
*I nici g 1 a separate base station. Communications between server .4 and 
apiece ,0 could a,so be imp.emented with radio signal, For example, app.tance 10 
could incorporate a conventional paging receiver. 
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A particular example of the embodiment illustrated in Figure 5 is a 
programmable telephone. Speed dial numbers and other programmable features of a 
telephone may be conveniently set using a graphical user interface hosted by server 14. 
Once the features have been programmed by the user, server 14 simply places a call to 
the telephone. Appropriate data demodulation circuitry is incorporated in the telephone 
in order to download the data from server 14. 

Figure 6 illustrates a further embodiment of the invention generally similar to 
that of Figure 5, but incorporating a transfer device as in the embodiment of Figure 2. 
Here, however, transfer device 16* receives data directly from server 14. As with the 
previously described embodiment, communication between server 14 and transfer 
device 16' may be telephonic or by radio. One example of a transfer device 16* is 
embodied as a removable module or "card" of a telephone. Data for an appliance 10 is 
downloaded from server 14 to the telephone where it is demodulated and stored in the 
card. The card may then be taken to appliance 10 to transfer the data to the appliance 
with an infrared or other data link. 

Another embodiment of the invention as illustrated in Figure 6 is a "universal" 
remote controller that may be coupled to a telecommunications network by means of an 
RJ-1 1 jack or equivalent in the manner disclosed by Darbee in U.S. Patent No. 
5,228,077. The remote controller could thus function as a data transfer device in 
addition to its more conventional remote control functions. 

As discussed above, the transfer device or appliance of the present invention 
preferably receives data by means of an opto-electronic data link. Any suitable source 
of light modulation may be employed to transmit data to the transfer device or 
appliance. These include LEDs, incandescent bulbs, LCDs and CRTs. A convenient 
source of light modulation is the display screen of a local computer. At least a portion 
of the display of the local computer may be modulated to transmit data to the transfer 
device. 

Most current approaches to video modulation data transfer use sequential 
pulsing of the video image to provide a series of binary l's and O's. These binary bits 
are used with framing bits (start and stop bits) to form complete data bytes. Some of 
the current approaches rely on the scanning CRT image to serialize the data bits by 
providing a luminance pulse for each data bit. This approach will fail when applied to 
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flat panel LCD screens because these screens do not have a scanning luminance 
response like that found with the CRT. 

Other methods provide a binary bit stream where each bit is produced at the 
vl deo field rate. For a typical CRT, this provides one binary data bit each 16 msec. (60 
fields per second). While this approach is viable for the CRT, it will not work well for 
flat panel displays. The slow response time of LCD panels mean that only a small 
number of data bits could be transferred per second. For a passive display, 3 b.ts would 
be possible (assuming 300 msec, response time). For active panels, 20 bits could be 
transferred Using conventional start and stop bits, a passive panel would then be 

capable of transmitting 0.3 bytes per second and active panels 2 bytes per second. Th,s 

is too low a data rate for many applications. 

Various modulation schemes are proposed below that are suitable for use with 

both CRT and LCD displays. Transmission of data from a remote server through a 

user's computer may be implemented with an appropriate applet written, for example, in 

the Java programming language. 

2 LUMINANCE MODULATION 

A first approach to data modulation of a display screen employs luminance 
modulation. This method drives the display with varying levels of intensity. Each 
tntensity level can represent an entire data digit. For example, Figure 7 illustrates a 
method using 10 shades of gray to represent a decimal digit. The luminance level or 
each successive decimal digit or luminance "dwell" is generated as fast as the display 
can accommodate. For an active matrix LCD panel, 20 dwells could be sent m one 
second This allows 20 digits of information per second, which is substantially faster 
than sending binary data. Higher data rates are possible with CRT displays, which are 
capable of transmitting a decimal digit in each video field. 

The luminance levels are detected by a photodetector in a receiving device. 
Discrete luminance levels can be generated using several different methods: 

a) Grayscales 

This method drives a spot on the display to one of several discrete shades of gray. 
The photodetector in the receiving device can detect the discrete levels and convert each 
level into a single digit value. Reference levels can also be sent periodically in the data 
stream to establish the black and white (highest / lowest) luminance levels. This allows 
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the photodetector output to be scaled to more accurately detect each discrete gray level. 
One limitation to this method is that the gray scale response of the display may not be 
linear. In fact, gray scale levels are greatly affected by the monitor's contrast and 
brightness controls. These controls can be changed by the user and are not predictable 
or known constants. 

b) Chromatic luminance 

It is possible to convey various luminance levels by selecting different color 
combinations. Each color has a luminance component combined with a chroma 
component. Selecting different colors also selects different luminance levels. For 
example, dark brown has a low luminance while cyan has a high luminance. Note that 
what is being detected with this method is luminance - not color. Accurate luminance 
detection depends on the color response of the display, the monitor contrast, brightness 
and color settings, and the color response characteristics of the photodetector. Accurate 
detection using this method typically requires some form of calibration to match the 
photodetector response to the display color response. 

c) Dithering 

With reference to Figure 8, the currently preferred method displays a regular 
pattern of black and white pixels within a region of the display to produce an average 
luminance level. This "dithering" average level is created by dividing the entire 
detection region into a matrix of smaller discrete cells comprising one or more pixels. 
Each cell is either driven full white or full black. The ratio of black to white cells 
determines the overall, average luminance for the detection area. This method 
eliminates problems with unpredictable gray scale response in the display due to 
contrast or brightness settings. 

The dithering approach illustrated in Figure 8 uses a rectangular matrix to 
obtain an average luminance for a given area. It is also possible to display other graphic 
patterns or characters which have a distinctive appearance while also presenting an 
average overall luminance. Some examples are shown in Figure 9. Each of these have 
a unique luminance level when the black areas are averaged with the white background. 
This allows the photodetector to discriminate between unique patterns or characters to 
convert to a corresponding data value. 
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d) Multi-color modulation 

Another method is to use two or more color channels to provide a means of data 
modulation. For this method two or more photodetectors are used, each responsive to 
different portions of the color spectrum. For example, separate red and green 
photodetectors could be used to detect varying shades of each color. Using two 
channels allows data encoding using the luminance level of each color channel, or the 
phase between two color signals. Phase modulation works by modulating the color 
channels at a given rate, but varying the phase relationship between the two channels as 
shown in Figure 10. 

To further increase the data density, it is possible to combine modulation of 
color luminance with color phase. Thus at any given sample interval, three parameters 
are available: red intensity, green intensity and phase relationship. If eight discrete 
values of each parameter are used, each sample point can represent 8 3 values or 1 of 5 12 
discrete numerical values per sample. A disadvantage to this method is the requirement 
for two color-selective detectors. Also, color response can vary between displays, so 
some type of color calibration may be required. 

3 video Bar Code 

Figure 11 illustrates another method of data encoding using video bar code 
modulation. This approach is similar to printed bar codes, but uses a higher density data 
coding With this method, a video bar code is displayed across the screen. The user 
swipes a receiving device across the bar code to read data from the screen. Conventional 
printed bar codes work by using different spaces between vertical lines. The spacing 
relationship is translated into binary data bits. Multiple bits are combined to form bytes 
of data. 

Using a video image, data can be represented using luminance levels or color. 
This allows higher data density because each "bar" in the video bar code can represent 
an entire decimal digit instead of just a single binary bit. This increases data density by 
8 to 10 times compared to conventional bar codes. 

Figure 11 illustrates a video bar code using luminance levels. Note that 
luminance levels can be generated using the same methods as previously described for 
spot modulation. Each bar represents one of many luminance levels, for example, with 
10 luminance levels each bar can represent a digit value of 0 to 9. 
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4. Color Modulation 

Chromatic luminance modulation was described above as a form of intensity 
modulation. It is also possible to employ a true color modulation in which specific 
color hues are used to represent corresponding numerical values. Depending on the 
range of hues used, an array of two or three separate detectors sensitive to different 
spectral components, such as by using appropriate filters, is required. A beam splitter 
may be employed to direct light to the individual detectors of the array in the receiving 
device. 

5. Self Clocking 

Regardless of the method of modulation employed, it is desirable that the data 
transmission be self-clocking. This means that individual data characters are detected 
by the receiving device without precise time spacing between characters. This self- 
clocking approach allows for pauses to occur during the transmission of data characters 
without creating transmission errors. Pauses can occur in PCs if the operating system 
performs another task while the transmission is active. For example, multitasking 
operating systems will commonly write data between memory cache storage and disk 
drives. This activity can preempt the operation of other software and cause short pauses 
in the operation of lower level applications. For internet based data transfers, varying 
delays are also common when moving data between servers and client PCs. 

It is also important to accommodate different data rates depending on the type of 
display monitor being used. Prior to starting the data transfer, the user can make a 
selection to indicate the type of display being used. If the display is a CRT, a faster 
transfer rate may be used (up to 75 digits per second). If an active matrix display is 
being used the transfer rate will be slower (20 digits per second). While the selection of 
transfer rate is easily accomplished on the PC side, the receiving device will preferably 
be compatible with all available transfer rates. Using self clocking data allows the 
receiving device to receive data at the transmission rate, without the need for a data rate 
selection on the receiving device itself. 

An efficient self-clocking method using a non-binary data encoding is illustrated 
in Figure 12. If luminance modulation is used the receiving device can detect each 
discrete luminance level change as a new digit. The length of time between successive 
digits is irrelevant. If the same digit value is sent twice in succession, a special "repeat" 
character can be used to indicate that the last digit value is repeating. As shown in 
Figure 12, 4 V indicates a repeating digit value. If the data stream the numerical value 
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1223 this would be transmitted as 1 2 > 3. More than two repeating digits are handled 
by re'sending the digit again after the repeat character. Thus, 122223 would be sent as 1 
2 > 2 > 3 With this approach a single digit value is never repeated twice in succession. 
The detector can simply wait for each change in luminance level to indicate a new digit . 
value has been sent. Timing relationships between characters is not significant. 

A self-clocking data interface in accordance with the present invention 
preferably uses a base 12 encoding system comprising decimal values 0-9, the repeat 
character ">" and a character to designate the end of a data record. Using this 
system a sequence of decimal values of arbitrary length can be transferred using, e.g., 
twelve'intensity levels. Alphanumeric data may be transferred using double digit 
values, each pair of digits representing the ASCII or equivalent code for an 
alphanumeric character. 

6 TIME INTERVAL MODULATION 

In contrast to self-clocking methods, another modulation approach is based on 
the time spacing between changes in intensity level or color. With this approach, only a 
limited number of intensity levels or colors is required. The number of discrete 
intensity levels or colors may be as few as two. The time interval between changes in 
intensity level or color has a number of possible discrete values, each of which 
corresponds to a numerical value. A significant advantage of this approach is that it is 
not sensitive to variations in display intensity or color fidelity. However, due to the 
characteristic response times, this approach is better suited to CRT displays than to LCD 
displays. 

7 RECEIVING DEVICE 

Figure 13 is a block diagram of a receiving device 200 suitable for use m 
connection with the present invention. Light emitted by (or reflected by) a display 
screen falls on photodetector 202. The output of the photodetector is amplified by 
amplifier 204 and asserted at the input of the analog-to-digital (A/D) converter 206. 
The digitized output, in this case comprising an 8-bit word, is presented as an input to 
microcontroller 208. The operation of microcontroller 208 is controlled by program 
instructions stored in read only memory (ROM) 210. These instructions govern the 
conversion of the raw digitized input from A/D converter 206 into a data digit. The data 
digits are further processed in accordance with the particular functions to be 
accomplished by receiving device 200. When configured as a transfer device, such as 
transfer device 16 discussed previously, receiving device 200 will further communicate 
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the data digits or information derived therefrom to a host device via a wired or wireless 
interface. A random access memory (RAM) 212 is coupled to microcontroller 208 for 
use as a scratchpad memory, the use of which may include temporary storage of the data 
digits received from A/D converter 206 or information derived therefrom. In many 
applications, particularly those related to transaction processing, receiving device 200 
will include a user interface 214 comprising a display and/or various controls, such as 
function selection buttons and the like. Receiving device 200 may also include a 
provision to allow for automatic calibration of the analog to digital converter. A peak 
detector 216 detects the peak white level in the received signal. This level is used to 
establish the upper range of A/D converter 206. This allows the full range of the A/D 
converter to be used over the receiver's data detection range. 

Receiving device 200 may be configured in any convenient form. As discussed 
above in connection with transfer device 16, receiving device 200 may have an 
elongated cylindrical shape similar to a pen or a wand. In such case, photodetector 202 
may be conveniently located at one end of the device. However, it has been found that 
pen- or wand-shaped devices have disadvantages when used with LCD flat screen 
displays. If the device is pressed against the display, even with light pressure, the 
display may be distorted, thereby affecting the accuracy of the data transfer. For flat 
panel displays, a flat, card-shaped receiving device is preferred. Such a device may be 
held against the display screen without distorting the display. 

To ensure proper registration of the receiving device with the display screen, a 
visual indication of the area of the display screen that will contain the data modulation 
is preferably provided. As shown in Figure 14, a rectangular area of the display screen, 
generally corresponding in size and shape to the card-shaped receiving device, may be 
configured as a window and may be labeled with a legend such as "place card here". A 
progress bar may be displayed adjacent to the transmission area to indicate the status of 
data transmission. 

In the examples discussed above, a single photodetector (or paired detectors in 
the case of bi-color modulation) is used in combination with a single modulated region 
of the display screen. It will be appreciated that the data transfer rate can be multiplied 
by employing a suitable array of photodetectors in combination with a corresponding 
array of data transmission regions. Obviously, the array of detectors must be properly 
registered with the array of modulated regions on the display. This can present a 
challenge in the case of a handheld receiving device. One solution to this challenge is 
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illustrated in Figure 14. Here, the display is divided into four quadrants that are 
independently modulated. The receiving device includes an array of four independent 
photodetectors. By providing simple registration marks on both the display screen and 
the receiving device, the receiving device can be held against the display screen so that 
the photodetectors are in proper registration with the corresponding quadrants. 

8. EXPERIMENTAL RESULTS 

A prototype system has been constructed. The prototype receiving device is 
configured as a card having the same length and width as a standard credit card. A 9 
mm round photodetector element is located in the center of one face of the card. 
Electronic circuitry within the card amplifies the output signal of the photodetector, 
which is then applied as an analog input to a conventional personal computer system 
where A/D conversion is performed. The photodetector element is designed to detect 
the average luminance over a 9 mm round area of the display screen. The detector 
consists of a translucent glass window and a photo-Darlington transistor photodetector 
mounted in a plastic enclosure. 

The prototype system employs luminance modulation using the dithering 
approach discussed above. A total of 12 luminance levels are used to represent ten 
decimal values plus two additional values to indicate formatting and repeating 
characters. Using a CRT display, the prototype system has achieved data transfer rates 
of 20 characters per second. 

The photodetector in the receiver detects the luminance change as the electron 
beam in the CRT passes over the detector. This screen phosphor glows with a 
brightness related to the average screen luminance. For a CRT display, the beam u 
constantly scanning the screen. This creates a pulse as the beam passes over the 
detector Therefore, the signal detected is a pulse which repeats at the frame rate ot the 
display (typically 13 to 17 msec, per field). Figure 15 is an actual capture of the signal 
received by the detector using a CRT-based display. 

Since the received signal is a pulse, a software algorithm processes the A/D 
conversion readings in order to establish the luminance level represented by the peaks 
of the detected pulses. The software algorithm is then capable of decoding these levels 
back into packets of data. 
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It is desirable for the system to automatically adapt to varying intensity levels on 
the display. Different luminance levels will result due to variances in the brightness 
response of the display, the sensitivity characteristics of the photodetector and also due 
to adjustment of the brightness and contrast settings of the monitor. 

To automatically adjust for these differences, the system provides a calibration 
sequence at the start of each data transmission. As illustrated in Figure 16, the 
calibration pattern consists of a staircase of each of the 12 luminance levels used. A full 
white pulse (level 12) is sent at the start of the sequence, followed by values of 0 to 12. 
This signal is detected by the receiver and used to establish the actual 12 discrete levels 
obtained from the monitor. In Figure 16, the bottom signal is the actual pulse 
waveform received by the photodetector. The top signal is that obtained after 
processing by a software algorithm. 

9. Further Embodiments of the Invention 

Another specific example of the utility of the present invention is illustrated in 
Figure 17. A transfer device 300 is essentially similar to device 200 described above. 
Data is transferred into device 300 from a local computer 302 as previously discussed. 
In this case, device 300 is loaded with information describing the contents recorded on a 
storage medium, such as videotape cassette 304. The information loaded into device 
300 may be inputted directly by the user at the local computer. Preferably, however, an 
interactive television program guide is employed so that the user can simply "point and 
click" on the particular programs that have been recorded. 

Transfer device 300 is then docked into videotape cassette 304. A display 306 is 
then employed to provide a visual index to the contents of cassette 304. Alternatively, a 
display 308 may be built into cassette 304, in which case transfer device 300 need not 
include display 306. In this case, the information describing the contents of cassette 304 
may be stored within the cassette itself so that the transfer device may be used to load 
information into a plurality of cassettes. 

Still another example of the utility of the present invention is shown in Figures 
18-20. Figure 18 illustrates a virtual companion 410 appearing on the display screen 
412 of a personal computer system. Virtual companion 410 may be created by a 
software program running on the computer system. Such software may be purchased 
and installed using conventional magnetic disks or a CD-ROM. The software may also 
be downloaded from a remote site. Alternatively, virtual companion 410 may be 
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oenerated by a software program on a remote server that is accessed by the personal 
computer system via the world wide web. The virtual companion may resemble a pet 
animal, a commercial doll or stuffed toy, a character from a book or comic stop, an 
historical figure, a sports personality, etc.. Preferably, certain attributes of the virtua 
companion, such as the companion's name, limited aspects of appearance, dress, voice 
and spoken phrases may be customized by the user with appropriate control panels and 
the like on the personal computer system. 

An integral part of the invention is handheld device 420. This device would be 
purchased by the user in conjunction with software for the personal computer system or 
with a password to a corresponding web site. Once the virtual companion has been 
configured according to the user's wishes, it may be "transported" from d 1S play screen 
412 to device 420. To accomplish this, the user initiates a transfer by clicking on a 
transfer icon 414 on display screen 412 or by other suitable input means. The user is 
then instructed to place device 420 over a location 416 on display screen 412. 

Referring now to Figure 19, virtual companion 410 appears on display screen 
422 of device 420 when the transfer is completed. Appropriate sound effects may be 
generated by the computer system during the transfer process. Virtual companion 410 
no longer appears on display screen 412 of the computer system. As indicated » 
Figure 20, ' -ice 420 may now be removed from display screen 412 and carried about 
with virtual companion 410 "resident" therein. 

Software installed in device 420 controls virtual companion 410 while it is in 
"residence" Such software would include at least a limited capability for animating 
virtual companion 410 and for enabling it to articulate words and/or sounds. The 

manner in which virtual companion 410 is controlled may be downloaded into device 
420 as part of the transfer process. 

When desired, virtual companion 410 may be retnmed to display screen 412. 
This transfer may again be initiated by clicking on an icon on the display screen or by 
other suitable input means. It is possible to achieve this transfer even if there ts only 
one-way communication from display screen 412 to device 420. When the transfer is 
initiated, data is sen. from display screen 412 ,o device 420 .0 cause virtual compamon 
410 to be removed from display screen 422. Simultaneously, virtual compamon 410 
appears on display screen 412. However, if device 420 is no. properly aligned w.«h 
,ocadon 416 on display screen 412, virtual companion 410 may appear on dtsplay 
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screen 412, even though it is not removed from display screen 422. Therefore, is may 
be preferable to implement at least limited two-way communications between device 
420 and the personal computer system. One way of providing communication from 
device 420 to the computer system is via an audible signal from device 420 that is 
received by a microphone in the computer system. Alternatively,^ wired interface to a 
microphone input jack or other input port of the computer system may be used. 

A variety of interactive activities may be constructed around the virtual 
companion. For example, the virtual companion may be a participant in a game played 
on the personal computer system. The virtual companion may then be transferred to 
device 420 as a means of escape from some danger, for transport to a different time or 
place in the story line of the game, for "regeneration" or the like. Virtual companions 
may even be transferred between users. Thus, one user may "send" his virtual 
companion to a friend, who can then interact with the virtual companion and transfer it 
to her own device 420. 

Apart from providing a portable "habitat" for a virtual companion, device 420 
may have additional play value. For example, device 420 can serve as a readout device 
for text messages that are sent from display screen 412 using the data transfer protocol 
of this invention. Device 420 thus acts as a "decoder" for otherwise unintelligible 
messages. 

It will be recognized that the above-described invention may be embodied in 
other specific forms without departing from the spirit or essential characteristics of the 
disclosure. Thus, it is understood that the invention is not to be limited by the foregoing 
illustrative details, but rather is to be defined by the appended claims. 



WO 00/43870 



24 



PCT/US00/01541 



CLAIMS 

WHAT IS CLAIMED IS: 

1 . A method for setting programmable features of an appliance comprising 
the steps of: 

providing an interactive site on a global computer network; 

establishing a connection with the interactive site; 

interactively setting programmable features of a virtual appliance at the 

interactive site; 

providing a transfer device separate from the user's computer having an 
input port and an output port; 

downloading set-up data from the interactive site to the input port of the 

transfer device; 

downloading the set-up data from the output port of the transfer device to 
a real appliance corresponding to the virtual appliance. 

2. The method of claim 1 wherein the input port of the transfer device 
comprises an optical sensor. 

3 The method of claim 2 wherein the step of downloading set-up data from 
the interactive site to the input port of the transfer device comprises modulating a 
portion of a display screen and sensing such modulation with the optical sensor. 

4. The method of claim 3 wherein the step of modulating the display screen 
comprises displaying varying shades of gray. 

5. The method of claim 3 wherein the step of modulating the display screen 
comprises displaying varying colors. 

6 . The method of claim 3 wherein the step of modulating the display screen 
comprises displaying varying patterns of black and white pixels. 

7. The method of claim 6 wherein the pattern comprises a regular pattern of 

cells. 
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8. The method of claim 6 wherein the pattern comprises an irregular 
graphic pattern. 

9. The method of claim 6 wherein the patterns of black and white pixels 
have a plurality of average intensity levels greater than two such that each of the 
patterns represents a corresponding data digit having a numerical base greater than two. 

10. The method of claim 1 wherein the output port of the transfer device 
comprises an infra-red transmitter. 

11. The method of claim 1 wherein the step of downloading set-up data from 
the interactive site to the input port of the transfer device comprises downloading the 
set-up data to a computer coupled to the input port of the transfer device. 

12. The method of claim 1 further comprising the steps of extracting 
information from the set-up data and sending said information to a manufacturer of the 
real appliance. 

13. A method for setting programmable features of an appliance comprising 
the steps of: 

providing an interactive site on a global computer network; 
establishing a connection between a user's computer and the interactive 

site; 

interactively setting programmable features of a virtual appliance at the 
interactive site; 

establishing a direct communication link between the interactive site and 
a real appliance corresponding to the virtual appliance, said communication link 
being independent of the user's computer; 

downloading set-up data from the interactive site to the real appliance. 

14. The method of claim 13 wherein the communication link is telephonic. 

15. The method of claim 13 wherein the communication link comprises a 
radio frequency link. 
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16 . A method for setting programmable features of a telephone comprising 

the steps of: , 
providing an interactive site on a global computer network, 
establishing a connection between a user's computer and the interactive 

interactively setting programmable features of a virtual telephone at the 
interactive site; 

providing a real telephone corresponding to the virtual telephone having 
data decoder for decoding data transmitted telephonically; 

the interactive site establishing a direct telephonic connection with the 
real telephone independent of the user's computer; 

telephonically downloading set-up data from the interactive site to the 

real telephone. 

,7 An interface device for communicating data to an appliance comprising: 
means for coupling the interface device to a global computer network to 

receive modulated data; 

a data demodulator; 

a data memory coupled to the data demodulator; 
a data output port coupled to the data memory ; 
control means for initiating transfer of data from the data memory to an 
appliance via the output port. 

18 The apparatus of claim 17 wherein the means for coupling the interface 
device » the global computer network comprises means for modulating a display screen 
of a local computer. 

19 The apparatus of claim 17 wherein the means for coupling the interface 
device to the global computer network comprises a telephonic connection. 

20. The apparatus of claim 17 wherein the interface device is a removable 
module of the appliance. 

21. The apparatus of claim 20 wherein the interface device comprises 
electronics for controlling the appliance during operation thereof. 
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22. An apparatus comprising: 

a computer having means for executing a first stored program and a 
display screen; 

means for wireless transmission of data from the computer; 

a handheld device having means for receiving said wireless transmission 
of data, means for executing a second stored program and a display screen; 

wherein said first stored program controls displays of an icon on the 
computer display screen and causes said icon to be removed from the computer 
display screen in associate with wireless transmission of first data; and 

where in said second stored program causes said icon to appear on the 
handheld device display screen in association with receipt of said first data. 

23. The apparatus of claim 22 wherein the means for wireless transmission 
comprises modulation of at least a portion of the computer display screen. 

24. A method of providing an interactive activity comprising the steps of: 

providing a computer program; 

executing the computer program in a computer having a display screen, 
said computer program controlling display of an icon on the computer display 
screen; 

providing means for wireless transmission of data from the computer; 

providing a handheld device having means for receiving said wireless 
transmission of data, means for executing a stored program and a display 
screen; 

transmitting first data from the computer; 
removing the icon from the computer screen under control of the 
computer program in association with transmission of the first data; 
receiving the first data in the handheld device; 

displaying the icon on the handheld device display screen under control 
of the stored program in association with receipt of the first data. 
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